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ABSTRACT 

Under s t and ing the distribution of achi eveinent 1 evel s 
of students' performance on the National Assessment of Educational 
Progress (NAEP) is aided through the use of the trilinear chart. This 
chart is described and its use illustrated with data from the 1992 
state NAEP mathematics assessment. The trilinear chart is used to 
portray three variables simultaneously expressed in the form of 
components of a total. It is drawn like an equilateral triangle, each 
side of which is calibrated in equal percentage divisions from 0 to 
100. It is shown that one can see readily the trends in performance 
for different demographic groups for all of the 44 participating 
jurisdictions simultaneously. Fifteen displays illustrate the 
analyses and portray .some trilinear charts. (Contains 6 references.) 
(Author/SLD) 
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Using Trilinear Plots For NAEP State Data 1 



Howard Wainer 
Educational Testing Service 



Abstract 



Understanding the distribution of achievement levels of students' performance on 
NAEP is aided through the use of the trilinear chart. This chart is described and its use 
illustrated with data from the 1992 state NAEP mathematics assessment. It is shown that 
one can see readily the trends in performance for different demographic groups for all of 
the 44 participating jurisdictions simultaneously. 



*This research was supported by contract number R999B40013 from the National Center for Education 
Statistics to the Educational Testing Service, Howard Wainer, Principal Investigator. I am pleased to be 
able to acknowledge this help. In addition I am grateful for the advice and comments made on these charts 
by John Mazzeo and Paul Barton, which has led to some improvements. 
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Using Trilinear Plots For NAEP State Data 



Introduction 

Student performance on the tests of the National Assessment of Educational 
Progress (NAEP) has been characterized through a formal judgmental procedure into four 
levels: Advanced, Proficient, Basic, and Below Basic. Although the definitions of exactly 
what kinds of proficiencies constitute each level vary by age and subject matter, these 
levels are now in use in the math and verbal assessments. Moreover, current plans 
anticipate that they will eventually permeate all of the NAEP testing areas. 

Because these performance levels are considered important for a variety of policy 
decisions and as a criterion based measure of school effectiveness, many tables are 
produced and printed that report the percentage of children that score in each of these 
categories for each NAEP jurisdiction. These tables are produced for each test separately 
for each of many demographic variables (e.g., ethnicity, sex, community type, parental 
education). A sample table is sho^ as display 1 ( Table 1.4 from 1992 NAEP Trial State 
Assessment) 



Insert Display 1 about here 



This table is complete and allows the easy extraction of any state's data, but aside 
from providing the National mean scores, it does not yield any intuitive view of the 
distribution of performance across all of the states. Such a view is important for example, 
in making comparisons among various demographic subgroups. It is toward providing 
such an effective display that this report is aimed, 

Data Structure 

Making plots of the average score for each state is an easy graphical task; stem- 
and-leaf plots (Wainer, 1994) work very well indeed. The more difficult design problem 
is displaying the four numbers that characterize the percentage of students who are at or 
beyond each of the achievement levels. Four numbers usually require a four dimensional 
display. This is often at odds with our very Euclidean perceptions. But are these data 
really four dimensional? Since the four achievement levels are mutually exclusive and 
exhaustive, the four numbers must sum to 100%. Thus given any three of the numbers we 
know the fourth exactly. Data that have this character are usually called a 'probability 
simplex* and a three dimensional display can completely display all four numbers. One 
such display is a tetrahedron in which each of the four apexes corresponds to 100% of a 
single achievement level. Each edge corresponds to everyone being at either of the two 
apexes that form the ends of that edge. Each face corresponds to everyone being at one of 
the three levels whose apexes form the three corners bf that face. Any point that lies in 
the interior of the tetrahedron is a mixture of the four levels, the proportions of that 
mixture are measured by the distance of the point from the four apexes. Thus the full four 
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variable problem has a graphical solution, that involves each state being a single point in 
3-space. Such plots are currently in use for some applications (see Ramsay, 1993), but 
require either building a 3-dimensional physical model or using motion on a computer 
screen to simulate 3-dimensions. A graph that can be plotted on a piece of paper would 
have far greater usefulness. 

A 2-dimensional graph would be easy if we could somehow reduce the number of 
achievement levels to three without losing too much information. Sadly for the enterprise 
of American education, but happily for graphical display, the level "Advanced" is 
practically empty. From Display 1 we can see that in no state are there more than 5% of 
the pupils at this high level, with the national average at 3%. In Display 2 is a plot that 
shows the percentages of 4th, 8th and 12th graders that performed at the advanced level 
in mathematics in the 1990 assessment. It is clear that for most purposes very little 
information would be lost if we combine Advanced and Proficient into a single category. 
If we do this we have a much more practically tractable display problem. 



Insert Display 2 about here 



A Trilinear Plot 

With only three variables that always sum to 100% we have reduced the display 
problem to one that can be managed on a single face of the tetrahedron described earlier. 
Such a display has been called a "Trilinear chart" (Schmid & Schmid, 1978) and has 
enjoyed some successful applications (for example, Coleman (1961, p. 14), and more 
recently Upton (1994) in a splendid portrayal of the changes taking place in the British 
Parliamentary elections). The trilinear chart is used to portray simultaneously three 
variables expressed in the form of components of a total. "The trilinear chart is drawn in 
the form of an equilateral triangle, each side of which is calibrated in equal percentage 
divisions from 0 to 100. The rulings are projected across the chart parallel to the sides in 
the manner of coordinates" (Schmid & Schmid, 1978, p. 150). Some of the essential 
characteristics of a trilinear chart are shown in Display 3. The components are labeled 
normal to their associated axes which run from each apex to the mid-point of the opposite 
baseline. 



Insert Display 3 about here 



Some examples 

While it is certainly true that the trilinear chart can naturally accommodate the 
individual state achievement level data, can it illuminate those data in a way that is 
currently either impossible or difficult? While its format suits the data perfectly, it has 
two counts against it. First, a fully explicated version (as in Display 3) is so full of axes 
and labels that there is hardly any room left for the data and their associated labels. 
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Second, it is not a commonly familiar display and so any prospective use will require 
some training for the reader. This may limit its applicability to a more technical audience. 

We can attempt to deal with the first problem by simplifying the display 
somewhat, removing many of the grid lines and use the display to show broad structures 
rather than allow the extraction of details (which are better done with tables in most cases 
anyway). The second problem is an empirical one. How much training is necessary to 
allow a naive reader to understand data presented as a trilinear chart? 

In Display 4 is a trilinear chart that shows the performance of the U.S. and two 
jurisdictiDns, Iowa and the Virgin Islands, on the 1992 8th grade NAEP State 
Mathematics Assessment. The arrows springing from each jurisdiction intersecting 
perpendicular to the three axes show how the points should be interpreted. 37% of Iowa's 
students performed at the 'Advanced or Proficient' level, compared to only 1% of those 
from the Virgin Islands and 23% for the nation as a whole. 44% of Iowa's students were 
at the Basic level compared to 12% in the Virgin Islands. Last, only 19% of Iowan's were 
'Below Basic' compared to 87% in the Virgin Islands. Obviously these are two extreme 
points, but they serve to illustrate how to read the chart. Intuition is aided by noting the 
direction and distance one must traverse in moving from the Virgin Islands to Iowa. In 
geographic terms, moving east is good, moving north is better; in general any jurisdiction 
that is northeast of another dominates the latter in both "Basic" and 
"Proficient+Advanced" categories. 



Insert Display 4 about here 



Showing two points in a comprehensible way is no trick. How well does this 
display method allow us to look at large quantities of data? Shown in Display 5 are all 44 
of the participating jurisdictions (41 states, Guam, the Virgin Islands and the District of 
Columbia), as well as small squares representing the four aggregate regions of the 
country as well as an open circle representing the national average. The general structure 
is clear, although it is difficult to identify any particular jurisdiction. As we can see all of 
the points range near a diagonal line with a slope of about 30°. None of the points fall into 
the top segment labeled 'Proficient & Advanced'. What does this mean? 



Insert Display 5 about here 



In Display 6 we have simplified the picture somewhat by omitting portions of the 
axes lines. In so doing we have formed and labeled three tridants (surely not quadrants). 
Any jurisdiction that falls into one of these tridants has that tridant as its modal level. 
Thus, for example, Iowa, Minnesota and North Dakota have more 8th graders at the 
Basic level than at either of the other two. Whereas Guam, Mississippi and Louisiana, 
have more 8th graders at the 'Below Basic' level than any other. It is discouraging to note 
that no jurisdiction's point falls into the 'Proficient and Advanced' sector. 
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Insert Display 6 about here 



We have seen how we can look at all three levels of achievement for all of the 
NAEP jurisdictions simultaneously. Can we use this same tool to compare various 
subgroups? YI?e answer is "certainly," but how can this be done most efficaciously? In 
Displays 7, 8, 9 ? and 10 are trilinear plots for all of the NAEP jurisdictions with large 
enough samples to yield estimates of acceptable accuracy for the four most common 
ethnic groups: Blacks, Hispanics, Whites, and Asian/Pacific Islanders. One way to 
display these is, as we have done, arrayed over a page. The eye can then wander around 
the cage and see the profound differences among the ethnic groups. We see the obvious 
outcome of Black and Hispanic students doing poorly; Whites and Asians doing 
considerably better. We can see that there is no overlap between the distributions of 
whites and blacks; that the state in which blacks do best still performs worse than the 
state that whites do worst. The plot for Asian/Pacific Islanders shows us many things, two 
among them are: only ten jurisdictions had enough Asians to yield any estimate at all, and 
that four states (Maryland, Connecticut, New Jersey and Texas) had Proficient and 
Advanced* as their modal level. 



Insert Display 7, 8, 9, 10 and 1 1 about here 



Another way to compare the performances of the various ethnic groups is to plot 
them all together on the same set of axes. This is useful when, as in the Black- White 
comparison, the distributions are non-overlapping. When this is not the case the resulting 
plot may become too busy. A dynamic alternative is to stack the plots one on top of the 
other on a computer screen (with only the top one visible) and switch from one to the 
next at the rate of perhaps one or two a second. The Phi-phenomenon takes over and all 
we see is a moving clow* of points within a fixed outer frame. A static alternative 
(Display 1 1) would be to substitute a suitably shaped and scaled oval for the cloud of 
points. In this version we can not only see the differences in location of the performance 
of the four ethnic groups represented, but also something of their variability. Note that 
The performance of Whites and Blacks seems to be more homogeneous than that 
exhibited by Hispanics and Asians. 

Conclusions and expansions 

Levels of Achievement are tabled for several other demographic variables besides 
ethnicity. Unlike ethnicity, some tuese variables can be thought of as causal. One such 
background variable is Parental education*. Can we use the trilinear plot to help us to 
understanc the approximate size of the causal effect of such a variable? To explore this 
we can plot the points associated with children whose parents did not graduate from high 
school (Display 12) and again for those whose parents are college graduates (Display 13). 
Viewed side-by-side we can see that there is very little over-lap between the two 
distributions. But to see the size of the effect of the variable "parents' education" we must 
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connect the points representing each jurisdiction. The size of the line joining them is a 
measure of the size of the effect of the variable "parents' education." 



Insert Display 12, 13, 14, and 15 about here 



In Display 14 is an attempt to do this with only ten states. It is too confusing to be 
of much help, although we do see the expected effeci chat as the bottom point shifts to the 
right the slope of the connecting line gets steeper. This isn't surprising. It merely shows 
graphically that in states in which a greater proportion of children from relatively more 
poorly educated families do rather well, those children from the same state from better 
educated families do better still While this is not a surprising result (in fact it is almost 
tautological), it is reassuring to see that even in as over-busy a display as this, it jumps 
out at us. 

One simplification that may be of help is to utilize this display technology within 
a dynamic framework. Most users of the display have particular interest in a specific 
jurisdiction. Thus we can simplify the display considerably by omitting all extraneous 
graph lines and plotting only the end points for each jurisdiction, keeping visually 
separate the symbols for the two groups of children from homes with vastly different 
backgrounds in schooling. Then connect only the points for the jurisdictions of interest. 
In Display 15 is one version of this. The is a line for New Jersey and another, principally 
for comparative purposes, for the entire country. The immediate message we get is that 
New Jersey's children on both extremes of this spectrum perform among the top 8 or 10 
jurisdictions. 

Simplifying the display by removing some of its explanatory elements is a 
reasonable thing to do once the reader becomes accustomed to its character. A graph as 
full of help as the one pictured in Display 3 leaves little room for the data. An important 
consideration whenever an innovative graphical format is proposed must be the gains 
associated the new form versus the losses associated with moving away from the 
conventional display. Pie charts are used despite their flaws because they are a 
conventional and obvious metaphor. Trilinear plots are not in common use, and despite 
their obvious appropriateness in this application, they take some getting used to. It was 
our intention to provide some experience with the 13 variations and examples here so that 
the reader can gain experience and comfort with the format. Once this has occurred the 
display components that orient the naive viewer can be downplayed visually leaving 
more room for data. One obvious direction for increasing the richness of the data would 
be an expansion of Display 14 by deleting the points entirely and substituting jagged lines 
for each jurisdiction that connect the (now invisible) points associated with several levels 
of parental schooling. How many lines can be visually accommodated is an empirical 
question. But the tale told by such a single image, were it comprehensible, would be 
nothing less than a state-by-state depiction of the effect of parents' education on children's 
mathematics performance. Being able to do this in an accessible way seemo a worthy 
goal. 
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Display Captions 



Display I. Excerpt from Table 1.4 from NAEP 1992: Mathematics Report Card for the 
Nation and the States showing the percentage of public school children at each of 
the achievement levels for the 44 participating jurisdictions, four selected 
aggregates, as well as for the nation as a whole. 

Display 2. A plot showing the relative percentages of 4th, 8th and 12th grade students in 
the 1990 Mathematics Assessment at each of the four achievement levels. It 
indicates that little information is lost by combining the category "Proficient" with 
that of "Advanced." 

Display 3. A fully detailed description of a sample trilinear chart from Schmid & Schmid 
(1978). 

Display 4. A sample trilinear chart showing the relative positions of Iowa, the Virgin 
Islands and the Nation on the 1992 NAEP State Mathematics Assessment. 

Display 5. A sample trilinear chart showing the relative positions of all participating 
jurisdictions on the 1992 NAEP State Mathematics Assessment, 

Display 6. A simplified trilinear chart showing the relative positions of all participating 
jurisdictions on the 1992 NAEP State Mathematics Assessment. On this version 
about half of the jurisdictions are explicitly identified and the principal tridants 
are graphically emphasized. 

Display 7. A sample trilinear chart showing the relative positions of all participating 
jurisdictions on the 1992 NAEP State Mathematics Assessment for Black 
students. 

Display 8. A sample trilinear chart showing the relative positions of all participating 
jurisdictions on the 1992 NAEP State Mathematics Assessment for Hispanic 
students. 

Display 9. A sample trilinear chart showing the relative positions or all participating 
jurisdictions on the 1992 NAEP State Mathematics Assessment for White 
students. 

Display 10. A sample trilinear chart showing the relative positions of all participating 

jurisdictions on the 1992 NAEP State Mathematics Assessment for Asian/Pacific 
Islander students. 

Display 1 1. A schematic trilinear chart showing the relative positions of all participating 
jurisdictions on the 1992 NAEP State Mathematics Assessment for all four ethnic 
groups. 



Page - 8 12 November 1 1, 1994 



Display IL. A sample trilinear chart showing the relative positions of all participating 
jurisdictions on the 1992 NAEP State Mathematics Assessment for children 
whose parents did not graduate from high school. 

Display 13. A sample trilinear chart showing the relative positions of all participating 
jurisdictions on the 1992 NAEP State Mathematics Assessment for children 
whose parents were college graduates. 

Display 14. A sample trilinear chart showing the relative positions of ten participating 
jurisdictions on the 1992 NAEP State Mathematics Assessment for children 
whose parents did not graduate from high school as well as for children whose 
parents were college graduates. The length of lines connecting u se points 
indicate the size of the contribution that parental schooling has on children's 
performance on 8th grade mathematics. 

Display 15. A sample trilinear chart showing the relative positions of all participating 
jurisdictions on the 1992 NAEP State Mathematics Assessment for children 
whose parents did not graduate from high school as well as for children whose 
parents were college graduates. The gain associated with parental education is 
depicted explicitly for New Jersey by joining the two points for that state. It is 
also done for the nation as a whole to provide a measure of comparison. 
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(1.0) 


2 


(0.5) 


ie 


(1.1) 


53 


(1.2) 


47 


(14) 


Tennessee 


256 


(1.4) 


1 


(0.4) 


15 


iW 


53 


(1.6) 


47 


(14) 


Texas 


264 


(1.3) > 


4 


(0.6) 


21 


(1.4) > 


56 


(1.5) > 


42 


(15). 


Utah 


274 


(0.7) 


3 


(0.5) 


27 


(1.1) 


72 


(1.3) 


26 


(14) 


Virginia 


267 


(1.2) 


3 


(05) 


23 


(1.2) 


62 


(1.6) 


36 


(1.6) 


West Virginia 


256 


(10) 


1 


(0.2) 


13 


(OJ) 


53 


M4J 


47 


(14) 


Wisconsin 


277 


(14) 


4 


(0.6) 


32 


d- 4 ) 


76 


(1.«) 


24 


(1.9) 


Wyoming 


274 


(0.9) > 


2 


(05) 


26 


(1.0) 


73 


(1.3) 


27 


(1.3) 


TERRITORIES 






















Guam 


234 


(1.0) > 


1 


(0.2) 


7 


(0.7) 


30 


(1.4) 


70 


(14) 


Virgin islands 


222 


(1.1)> 


0 


(0.1) 


1 


(0.3) 


13 


(1.0) 


67 


(1.0) 
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